Leveraging a rolling cross-section design in panel studies to investigate police legitimacy

Monica Gerber and Cristóbal Moya



London School of Economics and Political Science

May 22, 2023

Today

  1. Motivation
  2. The rolling cross-section design
  3. This Study: a rolling cross-section panel design
  4. Implementation challenges

Motivation

Panel studies

  • Confirmatory research
  • Individuals are interviewed at two or more points in time
  • Possible to make inferences about changes in attitudes or behaviours at the individual level

Motivation

Panel studies

  • Confirmatory research
  • Individuals are interviewed at two or more points in time
  • Possible to make inferences about changes in attitudes or behaviours at the individual level

Rolling cross-sectional design (RCS)

  • Exploratory research
  • RCS: distribute interviewing within a cross-section in a controlled (random) way over time
  • Capture potential impact of unexpected events

Unexpected event during survey design

“research design that exploits the occurrence of an unexpected event during the fieldwork of a public opinion survey to estimate its causal effect on a relevant outcome by comparing responses of the individuals interviewed before the event \(t_i<t_e\) (control group) to those of respondents interviewed after the event \(t_i>t_e\) (treatment group).” (Muñoz et al., 2020, p. 187)

  • Quasi-experiment: Effects of naturally occuring events that cannot be manipulated through controlled experiments
  • Assumption that interview time is independent from the time when the event occurs

The rolling cross-section design

The “expected-unexpected” event during survey design

“the design is just a standard cross-section, but the day on which a respondent is interviewed is chosen randomly” (Johnston & Brady, 2022, p. 283)

“The data collected in an RCS allow a fine-grained analysis of dynamic phenomena of public opinion” (Lutz et al., 2013, p. 3)

  • Temporal heterogeneity as an opportunity to move closer to causal inference (Johnston & Brady, 2002)

The rolling cross-section design

The “expected-unexpected” event during survey design

  • Steps (Kenski et al., 2010):
    1. Total number of interviews is established
    2. Total sample is divided into multiple replicates (smaller random samples)
    3. Replicates are assigned to be interviewed at different intervals of times
    4. All replicates are treated in the same way (protocol)

The rolling cross-section design

Example

The rolling cross-section design

Assumptions

  1. Temporal ignorability: “for any individual, the potential outcomes must be independent from the moment of the interview” (Muñoz et al., p. 189)
  • Moment of the interview should be as good as random
  • In RCS moment of interview is random
  • Protocol to ensure equal procedures to contact and recontact sample units
  • Need to compare ‘treated’ and ‘not-treated’ in terms of ease of contact and socio-demographics (Lutz et al., 2013)

The rolling cross-section design

Assumptions

  1. Excludability: “any difference between respondents interviewed before and after the event shall be the sole consequence of the event” (Muñoz et al., p. 189)
  • Timing of the interview \(t\) affects \(Y\) only through event \(T\)
  • Beware of collateral events, simultaneous events and unrelated time trends
  • Need to characterize the event and its reactions

The rolling cross-section design

Assumptions

  1. Compliance: “Assignment to the treatment group \(t_i>t_e\) was assumed to perfectly correspond to actually receiving treatment \(T=1\).” (Muñoz et al., 2020, p. 194)
  • Need to measure compliance

This Study

Police legitimacy

  • Police work requires public support to be able to function in a democratic context
  • Beliefs that the police act in procedurally just ways and are legitimate facilitate compliance and cooperation with authorities
  • A socially legitimate police force should require little violence to confront opposition
  • However, the world has seen many cases of violent demonstrations, police violence and human rights violations

This Study

The Chilean case

  • Chile’s main police force, the Carabineros, was among the most trusted institutions (Dammert, 2019)
  • Yet, recent events of corruption and excessive use of force during the social uprising in October 2019 have produced a serious loss of trust (Gerber et al., 2023)
  • Crisis of legitimacy: proposals of “deep reform” and even the dissolution of the institution
  • Need to understand relationship between police and citizens in Chile during the next crucial years

This Study

Aims

  • Confirmatory:
    • Track over time people’s perceived police legitimacy and motivational postures towards the police, and examine their relationships with perceived procedural justice and fear of the police in Chile
  • Exploratory:
    • Determine the possible effects of the occurrence of different events (such as police violence, corruption scandals or reforms) on perceptions of legitimacy, procedural justice and fear of the police in Chile.

This Study

A rolling cross-section panel design

  • Probability-based online panel of the Chilean population (18-65 years old)
  • Panel study with 10 waves
  • Sample size
    • Expected sample in contact survey: n = 5.300
    • Expected sample in Wave 1: n = 1.500
  • Possibility to combine with non-probabilistic sample to increase sample size

This Study

A rolling cross-section panel design


This Study

A rolling cross-section panel design


This Study

A rolling cross-section panel design


This Study

A rolling cross-section panel design


This Study

A rolling cross-section panel design


This Study

A rolling cross-section panel design

Simulación Cristóbal

This Study

Implementation challenges


This Study

Implementation challenges


Thank you!

  • Contact:
    • Monica Gerber (monica.gerber@udp.cl)
    • Cristóbal Moya (x)